The present invention is generally directed to systems and methods for scheduling jobs running on a data processing system. More particularly, the present invention is directed to providing user control of a wide range of computer related resources which exists at a plurality of levels within a data processing system. Even more particularly, the present invention is directed to the utilization of a consistent set of resource models extending over different data processing levels from a low hardware level through the operating system level and onto an administrative and user level.
With the proliferation of data processing systems employing multiple processors it has typically been the role of the operating system to schedule jobs based upon job descriptions provided by system users. In contrast, the present invention breaks away from this modality by providing mechanisms for much greater user involvement in the utilization of data processing system resources.
In particular, it is noted that the increased use of data processing systems employing multiple processors particularly in a symmetric multiprocessing (SMP) environment, there has been provided an opportunity for various tasks associated with a user""s job to be run on selected ones of a number of processing units. Accordingly, the present invention includes a mechanism for user controlled association of various job tasks with particular processors. Accordingly, this permits greater control of the data processing operation by an entity which is most familiar with the structure of the program or application, namely the user of application software. For example, users running application programs which are particularly susceptible to parallelization are provided with a specific mechanism for linking various job tasks with different processors.
While it is noted that the present invention is particularly directed to symmetric multi processing environments and to the allocation of processors as a data processing system resource, the present invention also extends to a more generic modality in which a number of different system resources are modeled and defined in a consistent way across a number of different system levels.
In particular, it is noted that resources such as the amount of memory available may be defined as being available to a user as a hardware level resource. It is also noted that resources such as virtual memory or the number of CPUs available may be defined and modeled at the operating system level so as to provide a mechanism for user control of these resources in a direct fashion. Additionally, there are resources available at the administrative level, a level which overlies the operating system and is available in such systems as the assignee""s IBM""s RS/6000 SP product employing LoadLeveler as part of this administrative level. Resources now available at this administrative level include such items as available software licensing. This might occur for example in a situation in which a desired licensed application program has associated with it a contractual obligation that the program may not be run on more than N processors at the same time under the given license. Likewise it may be the case that a software license is limited by the requirement that the software run only on specified (and/or identified) processing units.
Additionally, users wanting to run large simulation programs often need to be able to keep all of the relevant simulation data in memory at the same time in order for the application to run efficiently. However, the allocation of memory as a resource has typically only been accomplished through the operating system, and has not been under user control. Furthermore, some jobs actually have a requirement that more than one CPU is involved in the computation.
In addition to placing all of these computer resources within greater control of a user, it is also very desirable that any such scheduling is done in a dynamic configuration without any requirements for rebooting or otherwise intervening with operating system functions.
In order to best understand the proper context in which the present invention operates, it is noted that the present invention does not operate at the microsecond or nanosecond time scales of the operating system. Rather, the present invention operates as a part of a macro scheduler operating above the operating system level. Nonetheless through the utilization of API-like (Application Program Interface) statements, proper communication with the operating system level is established in a consistent manner using a resource model which is applicable over several data processing system levels. In particular, the model employed by the present invention are such that resources are specified in terms of both type and amounts. Thus, there is typically a quantity or capacity associated with each one of the resources which are modeled in the present invention.
In accordance with a preferred embodiment of the present invention a method for managing resources in a data processing system comprises an initial step of defining at least one type of resource together with a quantity which is associated with that resource type. This quantity is indicative of resource capacity. The resource is either a hardware level resource, an operating system level resource, an application level resource an a user level resource. A determination is made as to whether a user has requested the use of one of these resources. Then the availability of the requested resource is determined with respect to its indicated type and quantity. Finally a user job is dispatched in accordance with the available resource type and quantity.
The present invention functions by providing a consistent resource model which extends across multiple data processing systems levels (hardware through operating system through administrative and finally through user levels). The resource model provided herein is defined in terms of type and quantity. The model is invoked through the utilization of API like statements provided by the user (or their interactive equivalents). These statements are analyzed in order to determine the requested resource usage. These analyzed statements are then used to schedule and to run jobs which use the specified resources. The power and flexibility of the present invention derives from the fact that a range of resource types is modeled in a consistent fashion across a number of data processing system levels particularly with respect to the incorporation of type and quantity specifications and in the modeling of a large variety of resources which are contemplated as being handled in accordance with the present method.
Accordingly, it is an object of the present invention to provide users with greater control of how their applications are run.
It is also an object of the present invention to provide a better match between application programs and the resources which they require in a data processing system complex.
It is also an object of the present invention to permit a user to control the number of processors employed in the running of the user""s application program particularly in those environments in which there are a plurality of processors such as in a symmetric multi processor (SMP) architecture.
It is also an object of the present invention to permit users to control the type and amount of resources that a job consumes whether or not the resource is real or virtual.
It is a still further object of the present invention to provide a mechanism for accommodating constraints associated with limited software program licenses.
It is a still further object of the present invention to permit a user to control the amount of real or virtual memory used in processing a job or task.
It is also an object of the present invention to control the utilization of hardware resources by a user in a dynamic fashion which does not require either rebooting or operating system intervention.
It is a still further object of the present invention to provide a consistent resource model operable over a plurality of layers in a digital processing computer system or complex.
It is also an object of the present invention to improve the utilization of symmetric multi processing system environments.
It is also an object of the present invention to permit real time modification of resource requirements with respect to both type and to quantity.
Lastly, but not limited hereto, it is an object of the present invention to provide a mechanism for recovery of allocated and requested resources upon job and/or task completion.
The recitation herein of a list of desirable objects which are met by various embodiments of the present invention is not meant to imply or suggest that any or all of these objects are present as essential features, either individually or collectively, in the most general embodiment of the present invention or in any of its more specific embodiments.